The Avis Project: a Mixed Communication/interactive/broadcast Teleconferencing System

There is now no clear distinction between the three traditionally distinct service models: communication, interactive and broadcast. These service models can be mapped on the telecommunication, information technology and entertainment sectors, respectively. MPEG-4 is a proposed standard that supports all of these service models and consequently allows mixed models to be built. This paper presents the architecture of a mixed communication/interactive/broadcast teleconferencing system based on a dataaow model and MPEG-4 tools. The management of the distributed audio and video sources is done using an ATM network. Faces of the participants to the conference are analyzed in real-time using computer vision techniques and clones are locally synthesized using computer graphics tools. Video and audio spatialization are based on experience drawn from the entertainment sector. Immersive aspects are implemented using MPEG-4 tools. 1 Motivations and goals There is a great success of multiuser activities on the Internet, such as multi-participant adventure games and Internet Relay Chatting (IRC) services. Despite a poor or, at most, limited graphics interface, these systems are becoming more and more popular. One might see there a sign for users'indulgence of poor graphics quality but a deenite need for interacting with other participants. On the contrary, videoconfer-encing systems are often considered as boring, because of static images, the inability to interact with others participants and poor sound quality. Sound will become extremely important to reinforce graphics and make the conference more immersive. Positional 3D audio processing will have a signiicant impact on the acceptance of videoconferencing systems. We can even think that real-time positional sound is going to cause sound to surpass the present importance of graphics in multimedia projects. As in real life, sound is used to help the eyes know where to focus. Concurrent audio streams are interesting to make the conference more realistic. Our goal is to produce a framework that will allow for easy implementation and experiments of a video-conferencing system with the following characteristics: Spatialization of video This amounts to introducing a narrative point of view in coexistence with the usual active participation of VR systems. We want to construct smart (synthetic or not) cameras whose behaviour is described as an optimization problem. This non-convex optimization problem arises from : Perceptive aspects handled by projective constraints in the image plane Cognitive aspects handled by gaze and other 3D constraints Spatialization of audio In contrast to video, the perception of audio is omnidirectional. So video …